To decline or deteriorate.
They haven't been maintaining it, and the downtown area has really gone to pot over the past 20 years.
Go to hell. (an angry dismissal)